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(54) Imago translormation apparatus 

(57) The image captured by a video camera 2 mounted at a fixed position is transformed to Sj^^^ ^^^^^^^^^^^^^^ 
camora by rotation of the image plane modelled in a S^imensional space. The model of the ^^^^'^ ^^^^^^^ 
dependence on the angle of the camera with respect to the object W to be imaged Input means 1 ^ "^^^^f^^^^^ 
viwS. and cause memory controlling means 8 to carry out appropriate mappmg of image 

ToTut mernory 1 3. CorrecJion means 12 may apply distortions to selected parts of the image, eg to compensate for the 
difference in appearance of a tree from above and from the side. 
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TITLE OF THE INVENTION 

IMAGK PROCESSING APPARATUS 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to en image 
processing apparatus and more particularly to an image 
processing apparatus adapted such that the position of 
the visual point with respective to the picture image 
picked up with a television camera and displayed on a 
monitor is optionally changeable. 

Description of the Prior Art 

In television broadcasting or the like, to 
enable an object on the ground to be viewed from a 
plurality of positions of visual point, it has so far 
been practiced to dispose the television camera at high 
places with the use of a scaffold for picture taking or 
a crane so as to change the image pickup position of the 
television camera and hence the position of visual 
point. By relying on such con vent ional means alone, 
however, it sometimes becomes impossible to change the 
position of visual point because the crane or the 
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scaffold for picture taking becomes unable to be 
installed due to the conditions at the. place where the 
picture is taken. ■ 

OBJECTS AND SUMMARY OF THE INVENTION 
Objects 

A primary object of the present invention is 
to provide an image processing apparatus in which the 
position of visual point with respect to an object on 
the ground displayed -on a monitor is made optionally 
changeable without changing the image pickup position of 
the television camera. 

Another object of the present invention is to 
provide an image processing apparatus in which 
ujinaturalness to be produced in the picture image when 
the position of visual -point is moved with respect to 
the object displayed on a monitor is made correctable. 

Summary 

To achieve the primary object of the present 
invention, this invention, in an image processing 
apparatus in which the position of visual point with 
respect to the picture image displayed on a monitor is 
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movable, produces a model of the picked-up image of a 
surface of the object in a three-dinenfi ional coordinate 
system based on the coordinates of picture elements on 
the image pickup device of the camera and the image 
pickup angle of the camera with respect to the object,- 
maps the picture data output from the camera onto the 
model produced as above, and applies rotational 
transformation to the model with the video data mapped 
thereon in the three-dimensional coordinate system, and 
thereupon, displays the model with the picture data 
mapped thereon. 

To achieve the another object of the present 
invention, this invention, in an image processing 
apparatus in which the position x>f visual point with 
respect to the picture ijnage displayed on a monitor is 
m^ovable, deforms an area at a desired position in the 
model produced in a three-dimensional coordinate system 
based on the coordinates of picture elements on the 
image pickup device of the camera and the image pickup 
angle of the camera with respect to the object to 
thereby corrects the unnatur alness in the displayed 
picture image. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diBgraxn showing en 
embodiment of neans for image processing of the present 
invention; 

FIG. 2 and FIG. 2 are schematic diagrams 
explaining the principle of production of a planar model 
in a three-dimensional coordinate systejn performed in 
the means for image processing of the present invention; 

FIG* 4 is a ..sci^ratixac diagram showing an 
example of a picture image picked up by the camera in 
the means for image processing of the present invention; 

FIG* 5 is a s'cheroatic diagram showing a 
picture image shown in FIG. 4 with respect to which the 
position of visual point has been moved in the means for 
image processing according to the present invention; 

FIG. 6 shows schematic diagrams explaining the 
principle of correction* of unnatural portions in the 
picture image shown in FIG, 5; 

FIG. 7 is a schematic diagram showing a 
picture image after the unnatural portions have been 
corrected in the picture image shown in FIG. 5; 

• FIG. 8 and FIG. 9 are schematic diagrams for 
explaining the principle of production of a model having 
ups and downs in a three-dimensional coordinate system 
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performed in the means for image processing according to 

the present invention; 

FIG. 10 is a schematic diagram for explaining 
the portion becoming unnatural after the position of 
visual point has been moved in the means for image • 
processing according to the present invention; and • 

FIG. 11 is a flow chart for explaining manner 
of operation in the means for image processing shown in 
FIG. 1. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Structure of 'the image processing apparatus 1 
according to the present invention will be described 
below with reference to FIG. 1. 'An image of an object 

■9 nicked UP by a television 
such as a golf course 3 is pxcKca up uj 

c'amera 2. At this time, the television camera 2 is 

fixed and the image pickup angle is 6 • The image 

pickup angle is measured by an image pickup angle 

„easuring means 4 and supplied to a planar model 

producing means 5. The planar model producing means 5 

produces a planar model based on the input image pickup 

angle Q l The data representing the planar model 

produced by the planar model producing means 5 is 

supplied to a height information adding means 6. The 
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height information adding means 6 modifies the data 
representing the planar model based on height 
information delivered from a height information input 
means 7 to thereby produce data representing a model 
having ups arid downs. The data representing the model, 
having ups and downs is supplied to a memory controlling 
means 8. A video signal output from the television 
camera 2 is supplied to an input image memory 9 and 
stored therein such that picture elements are each 
stored at predetermined addresses. Meanwhile, a visual 
point setting means 10 produces visual point setting 
data based on visual point controlling data received 
from an input means 11 and supplies the visual point 
setting data to the memory controlling means 8« A 
correction means 12 produces correction data based on 
cofrection control data received from the input means 11 
and supplies the correction data to the memory- 
controlling means 8. The memory controlling means 8, 
based on the data representing the model having ups and 
downs supplied from the height information adding means 
6, the visual point setting data supplied from the 
visual point setting means 10, and the correction data 
supplied from the correction means 12, produces read 
addresses for the input image memory 9 and write 
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addresses for an output i»age »e„ory 13. The read 
addresses for the input in>age »emory 9 and write 
addresses for the output i»age »e»ory 13 are 
respectively supplied to the input inage «e«.ory 9 and 
output i»age »e»ory 13. whereby the video signal stored 

in the input i=age .e^ory 9 is read and stored at 
predeter»ined addresses of the output i»age »e»ory 13. 

The video signals read out of the output iioage »e»ory 13 

is supplied to a monitor 14 and displayed on its monitor 

screen- 

NOW. in the in>age processing apparatus 1 of 
the present invention picking up an i=age of a golf 
course 3 or the like with the television ca.era 2. the 
principle of its producing a planar ^odel in a three- 
dimensional coordinate sy.te. based on the i.age pickup 
angle 6 at that tin.e and capping a video signal output 
fro. the television camera 2 onto the planar model 
disposed in the three-di.ensional coordinate system will 

be described below. 

FIG. 2 shows a state, when an image of an 
object on a plane P is picked up by a television camera 
2 set in the direction perpendicular to the plane P. of 
the object on the plane P per spect i vely transformed onto 
the image pickup device (screen 20) of the television 
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cemcra 2. Conversely speaking, the real position of the 
object whose image is picked up by the television camera 
2 is on the plane P located at a specific distance from 
the screen 20 in the direction opposite to the position 
of visual point 21. Therefore, when the position OP of 
the object on the plane P is represented by (x, y, z), 
the position of visual point 21 is taken as the origin 
(0, 0, 0), the center C of the screen 20 is represented 
by (0, 0, SCP), and the position SP of the object 
perspectively transformed onto the screen 20 is 
represented by (X, Y, SCP), the relationship between the 
position OP of the object on th'e plene P and the 
position SP of the objett perspectively transformed onto 
the screen 20 is expressed by the following equations 

SCP * X 

>: = . (1) 

2 

4 

SCP * y 

V = . (2) 

2 

Then, a plane P* which is an inclined place of 
the plane P shown in FIG, 2 by a predetermined angle as 
shown in FIG. 3 is considered. Assuming that this plane 
P* is inclined by an angle 6 with respect to x - z 
plane at position M, the relationship between the 
position OP of an object on the plane P' and the 
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position SP of the object perspecti vely transformed on 
the screen 20 is expressed by the following equations 

' SCP * X 

X = = , . • . (3) 

2 + y cos 6 



SCP * y * sin e 
Y ^ . ... (4) • 

z + y cos e 

By solving the equations (3) and (4) for x and 
y , we obtain 

X * Z 4- X * y * cos Q 
^ = , ... (5) 

SCP 

• Y <^ 2 

y — ———————— ^ ————— ———— , # • . (6) 

SCP » sin e " - V » cos 
And then, z becomes 

2 = Z + y :f cos e . . ... (7) 

As apperent from equations (5), (6), and (7), 
by' measuring the image picP.up angle Q of the television 
cemere 2 with respect to the plane P' at the time when 
the image of the plane P' is picked up with the 
television camera 2, and substituting the angle 6 and 
the position (X, Y, SCP) of each picture element of the 
object perspecti vely transformed onto the screen 20 into 
equations (5), (6), and (7), the plane P' in the three- 
dimensional coordinate system can be calculated. Here, 



9 



2 is the distance from the position of visual point 21 
to the position M on* the plane P*. This distance is 
equal to the distance between the point at which the 
optical axis of the canera 2 intersects with the ^olf 
course 3 and the camera. Hereinafter, the plane P* 
produced as described above will be called "planar model 

By assigning video data of picture elements of 
the object perspect i vely transformed onto the screen 20 
to the corresponding coordinates on the planar model P* 
produced as described above, the mapping of the video 
data onto the planar model P' in the three-dimensional 
coordinate system is achieved. Thus, it is made 
possible to produce in the three-dimensional coordinate 
system the planar model P', which is inclined with 
respect to x - z plane by the angle the same as the 
image pickup angle G of the television camera 2 when 
picking up the image of the object on the plane P and 
has the video data. The video data mapped onto the 
planar model P* disposed in the three-dimensional 
coordinate system is supplied to the monitor 14 shown in 
FIG. 1 to be displayed thereon. 

As described above, by adding arbitrary 
values, which are mutually associated, to the coordinate 
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values corresponding to the positions of the picture 
elements of the planar Bodel ?' disposed in th^ three- 
dimensional coordinate system, the planar model P' can 
be rotationally transformed in the three-dimensional 
coordinate system. And thereby, it is made possible to 
change the position of visual point «ith respect to 'the 
planar model P* displayed on the monitor 14. 

Therefore, even if the image pickup angle 6 of 
the television camera 2 with respect to the golf course 
3 is fixed as shown in FIG. 1. the position of visual 
point with respect to the golf course 3 displayed on the 
monitor 14 can be changed. Hence, it is made easy to 
change the position of visual point such that as if the 
television camera 2 were brought, high to pick up the 
image of the golf course _3 from right above. More 
particularly, when there is picture information for 
example, as shown in FIG. 4 as a picture image picked up 
by a television camera 2. a picture for which the 
position of visual point is moved to a place high above 
as shown in FIG. 5 can be obtained without changing the 
image pickup angle 6 of the television camera 2. 

. Thus, according to the present invention, the 
position of visual point can be determined at will, and 
it is thereby made easy to acquire the sense of 
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distance. 

When the position of visual point is 
optionally changed as described above, however, since 
the planar model P' is formed with the object such as 
the golf course 3 approximated to a plane, those parts 
that are not lying on the plane P' such as trees W and 
hunans T in the picture image make the image unnatural 
when the position of visual point is moved. 

The state of an image of such an object as a 
tree W standing on the ground Q picked up with a 
television camera 2 from a position of visual point 21 
as shown in FIG. SA being displayed on the monitor 14 is 
shown in FIG. 6B. If, as shown in FIG. 6C, the position 
of visual point 21 with respect to the object displayed 
on the monitor 14 is brought to the position above the 
object as described above, the length of the tree W will 
be prolonged as shown in FIG. 6D. This is because the 
condition of the television camera 2 picking up the 
image of the object as shown in FIG. 6A is nothing but 
the condition, as shown in FIG. 6C, of its picking up 
the image of the tree W lying on the ground Q shown in 
FIG. 6A. . 

The method to correct the length of the 
standing tree W or human being T will be described 
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below. As the means for correction, the technique 
proposed earlier by the present applicant and described 
in U. Patent No, 4,791,581 is used. 

First, an area VCF to be deformed is defined 
within the planar model F* as shown in FIG.'SC and FIC. 
6D. Then, a deformation vector having direction, and 
quantity of deformation is set up. Further, a point of 
action CPi indicating the position to apply the 
deformation and a vector field function Fi are set up. 
Then, representing the position vector of the .planar 
model P' before the deformation by Pq, the position 
vector Pk of the planar model P' after the deformation 
is given by 

N 

Pn = PO + 2 Vi * Fi(Pi-i, CPi). ... (8) 
i = l 

The state of the planar model P' deformed as 
above is shown in FIG.-6E. That is, the tree W is now 
set upright. Thus, as shown in FIG. 6F, the prolonged 
state of the tree W is corrected, and thereby, the 
ground Q and the tree are displayed on the monitor 14 in 
the same state as that where the television camera 2 is 
brought to the position right above the tree. 

By performing the correction as described 
above, the picture image having humans T and trees K 
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made longer than real size can be corrected and thus the 
picture as shown in FIG. 7 can be obtained. 

Now, referring to FIG. B, the case where 
heights h(u, v) of an object such as the ground 0* are 
given by such an information source as a topographical 
map will be described. First, as described above, by' 
substituting the image pickup angle $ of the television 
camera 2 with respect to the ground 0' and the positions 
(X, Y, SCP) of the picture elements of the object 
perspectively transformed onto the screen 20 into 
equations (5), (6) and (7), a planar model P" in the 
three-dimensional coordinate system is obtained as shown 
in FIG, 2. Then, the thus produced planar model P" is 
deformed based on the data h(u, v) representative of the 
height at an arbitrary position (x. y, z) of the planar 
model P*". The coordinates . (x , y, z) of an arbitrary 
position in the three-dimensional coordinate system of 
the planar model P" after the deformation are calculated 
f rom 

>: 3r 2 + X * y * cos e 

y = ^ ... (9) 

SCP 



y * 2 

y = * : 

SCP * sin e - y * cos e 



+ h(u, v) * cos ^ , . , . (10) 



2 = Z + y * cose + h(u, v) * sine . ... (11) 

These equations can be derived from equations 
(6) and (7) by adding .components along y axis and z axis 
of the height h(u, V) of the object to their right-hand 
sides, respectively. By using the above mentioned 
equations (9). (10). end (11). a model p' ' ' having, ups 
and downs corresponding to the object in the three- 
dimensional coordinate system as shown in FIG. 9 can be 
produced. Further, by the use of equations (9). (10). 
and (11). it is made possible to uniquely associate 
positions of the picture elements of the object 
perspectively transformed onto the screen 20 with 
coordinates of the model" p'" having ups and downs. 
Therefore, by assigning video data of the picture 
elements of the object perspectively transformed onto 
the screen 20 to the corresponding coordinates of the 
model p"' having ups and downs, it is achieved to map 
the video data onto the model p' ' ' having ups and downs 
in the three-dimensional coordinate system. Thus, the 
model p'" having ups and downs which is inclined with 
respect to x - 2 plane by the same angle as the image 
pickup angle 6 of the television camera 2 when picking 
up the image of the object on the ground Q' and has the 
video data can be produced in the three-dimensional 
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coordinate system. The video data of the model p*'' 
having ups and downs disposed in the three-dimensional 
coordinate system 'is supplied to the monitor 14 shown in 
FIG. 1 and displayed on the same. 

As described above, by adding optional values 
which are mutually associated to the coordinates 
corresponding to positions of the picture elements of 
the model p"** having ups and downs and disposed in the 
three-dimensional coordinate system, the model p*'* 
having ups and downs and disposed in the three- 
dimensional coordinate system can be rotated. That is, 
the angle of inclination of the model p' ' ' having ups 
and downs with respect to x - 2 plane can be optionally 
changed. Therefore, when the video data of the model 
p*'' having ups and downs and arranged in the three- 
dimensional coordinate system is displayed on the - 
monitor 14, the orientation of the displayed model P' * ' 
having ups and downs can be changed. In other words, 
the position of visual point with respect to the model 
P'** can be changed. 

At the time the position of visual point with 
respect to the model P'*' having ups and downs is 
changed as described above, if there is a tree W on the 
ground Q' as shown in FIG. 10, the length of the tree W 
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will become too large when the position of visual point 
is brought high above the sa»e. This is because there 
are generally not included data expressing heights of 
trees W in the infornetion of the heights of the ground 
obtained from a topographical »ap or the like, and \ 
therefore, when producing the »odel P"' having ups' and 
downs, the heights of trees K are neglected and they are 
considered to be lying on the ground having ups and 
downs. 

The length of the tree K becoming too large 
when the position of visual point is brought high above 
the same can be corrected in the same way as described 
above using FIG. 6. First, an area VCF to be deformed 
is defined in the model P" ' having ups and downs. 
Then, a deformation vector Vi having direction and 
quantity of deformation is set up. Further, a point of 
action CPi indicating the position to apply the 
deformation and a vector field function Fi are set up. 
Then, by representing the position vector of the model 
P'" having ups and downs before the deformation by Pq. 
the position vector Pk of the model P" ' having ups and 
downs after the deformation is calculated by the use of 
equation (8). Thereafter, the model P'" having ups and 
downs is further deformed according to the position 
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vector Pn and thereby the length of the tree W can be 
corrected. 

Khen the! heights h(u. v) of the object such as 
the ground Q' are given by an inforoation source such as 
e topographical »ap, by producing a model P* ' ' having • 
ups and downs as described above and napping the video 
data onto the same, the heights of the ground after the 
position of visual point has been changed can be 
correctly displayed on the monitor 14. 

Below will be described manner of operation of 
the image processing apparatus 1 of thepresent 
invention shown in FIG.' 1 with reference to FIG. 11. 

At step SPl the image processing apparatus 1 

starts to operate. 

At step SP2. the image pickup angle measuring 
means 4 measures the image pickup angle 6 of the 
television camera 2 with respect to the object such as a 
golf course 3. The image pickup angle 6 may also be 
measured manually by the operator. The image pickup 
angle 6 measured through the image pickup angle 
measuring means 4 is supplied to the planar model 

producing means 5. 

At step SP3, the planar model producing means 
5 produces a planar model P* in the three-dimensional 
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coordinate system by substituting the image pickup angle 
e and the positions (X. Y, SCP) of the picture elements 
of the object pers^ectively transformed onto the image 
pickup device of the television. camera 2 into equations 
(5). (6). and (7). The data representative of the 
planar model P* calculated in the planar model producing 
neans 5 is supplied to the height information adding 
means 6. 

At step SP4. it is determined whether or not 
there is information about the heights (ups and downs) 
of the golf course 3 or the like. When it is determined 
that there is the information concerning the heights at 
step SP4. the data representative of the heights are 
input through the height information input means 7 to 
the height information adding means 6 at step SP5. The 
height information adding means 6 adds the data 
representing the height to the data representative of 
the planar model P' calculated in the planar model 
producing means 5. as indicated in equations (9). (10), 
and (11). When it is determined that there is no 
information about the heights at step SP4. the data 
representative of the planar model P' calculated in the 
planar model producing means 5 are directly supplied to 
the memory controlling means 8. 
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At step SP6, the loemory controlling means 8, 
based on the data supplied from the height inforoation 
adding means 6 or .the planar model producing means 5, 
produces read addresses for the'input image memory 9 and 
write addresses for the output image memory 13 and 
supplies these addresses to the input image memory 9 and 
output image memory 13, respectively. The video signal 
supplied from the television camera 2 end stored in the 
input image memory 9 is read out according to the read 
addresses supplied from the memory controlling means 8, 
and written into the output image memory 13 according to 
the write addresses supplied from the memory controlling 
means 8. Through this process, video data of .picture 
elements of the object perspect ively transformed onto 
the screen 20 can be assigned to the corresponding 
coordinates of the planar model P' or the model p»«' 
having ups and downs. This operation is the mapping. 

At step SP7, the data representing the 
deformed area VCF, data representing the deformation 
vector Vi indicating the direction and quantity of the 
deformation, and data representing the point of action 
CPi indicating the position where the deformation is 
performed and the vector field function are supplied 
through the input means 11 to the correction means 12. 
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Also, to the sane are supplied the data representing the 
position vector Pq -of the planar model P' before the 
deformation or the data representing the position vector 
Po of the model P' ' ' having upe and downs before the 
deformation from the memory controlling means 8. The . 
correction means 12. responsive to the incoming data, 
calculates the position vector Pk of the planar model P' 
after the deformation or the position vector Pn of the 
„odel P-" having ups and downs after the deformation by 
the use of equation (8). The deta representing the 
calculated position vector Pn of the planar model P' 
after the deformation or position vector Pn of the model 
p... having ups and downs after the deformation are 
supplied to the memory controllihg means 8. The memory 
controlling means 8. based on the data representing the 
position vector Pn supplied from the correction means 
12, produces the read addresses for the input image 
memory 9 and the write addresses for the output image 
memory 13 and supplies these addresses to the input 
image memory 9 and the output image memory 13. 
respectively. The video signal stored in the input 
image memory 9 is read out again according to the read 
addresses supplied from the memory controlling means 8 
and written into the output image memory 13 according to 
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the write addresses supplied fron the memory controlling 
neans 8. Through the described process the planar nodel 
P' or the model P'j' • having ups and downs can be 
adjusted so that the lengths of -the humans T and tress W 
are corrected. 

At step SP8, it is determined whether or no^ 
the correction at step SP7 has been completed. The 
process at this step 8 is performed by the operator 
determining whether or not the picture image displayed 
on the monitor 14 has been brought into a desired state. 

At step SP9, visual point controlling data is 
supplied through the input means 11 to the visual point 
setting means 10, and the visual point setting means 10 
in turn adds the values corresponding to the visual 
point controlling data to the coordinates of the planar 
model P' or coordinates of the model P'" having ups and 
downs supplied from the memory controlling means 8 to 
thereby produce visual point setting data. The visual 
point setting data is supplied to the memory controlling 
means 8. The memory controlling means 8, based on the 
visual point setting. data supplied from the visual point 
setting means 10, produces the read addresses for the 
input image memory 9 and the write addresses for the 
output image memory 13 and supplies these addresses to 



22 



the input ir.age memory 9 end the output image memory 13, 
respectively. The video signal stored in the input 
i„Bge memory 9 is read out again according to the read 
address supplied from the memory controlling means 8 and 
written into the output image memory 13 according to the 
write address supplied from the memory controlling ibeans 
8. Through the described processi the position of 
visual point with respect to the planar model P' or the 
model P"' having ups and downs can be changed. 

At step SPIO. the operator determines, from 
the display on the monitor 14. whether or not a desired 
state is attained as the result of the setting or change 
of the position of visual point with respect to the 
planar model performed at step SP9. When the desired 
state is attained, the process is ended at the following 
step SPU. That is. the setting of the position" of 
visual point and the correction of the picture image are 
performed between the operator and the image processing 
apparatus 1 in an interactive manner. 

According to the present invention, even if 
the image pickup angle of the television camera 2 with 
respect to the golf course 3 and the like is fixed as 
shown in FIG. 1. the position of visual point with 
respect to the golf course 3 displayed on the monitor 14 
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can be optionally changed. Therefore, it is made easy 
to change the position of visual point such that as if 
the television camera 2 were brought high to pick up the 
image of the golf course 3 from right above. Thus, the 
sense of distance can be easily acquired. 

Further, when the heights h(u, v) of the 
object such as the golf course are given by such an 
information source as a topographical nap, the model 
P'*' having ups and downs is produced as described above 
and the video data is napped thereon, whereby,- even if 
the position of visual point is changed, the heights of 
the ground can be correctly reproduced on the monitor 
14. 

Although the golf course was used as an 
example of the object in the above description of the 
embodiment, various other places such as baseball 
grounds can of course be used as the object in the 
present invention. 

Further, a picture image obtained by napping 
image data, which is obtained by picking up the image of 
a golf course or the like, onto a planar model produced 
by approximating the golf course or the like to a plane 
nay be combined with a picture image obtained by mapping 
image data, which is obtained by picking up the image of 

2A 



bu»8ns. trees, end the like, onto a pUnar .odel 
produced by approKi.ating the trees. hu»ans. and the 
like to a Plane, and thereby, a »odel of the golf course 
or the like having the trees, hymens . or the like »ay be 
produced in a three-dimensional coordinate systen,. 
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WHAT IS CLAIMED IS: 

1. An apparatus for processing image capable 
of moving the position of visual point with respect to 
an image displayed on a monitor (14) comprising: 

a. a camera (2) for picking up an image of an 

object (3); 

b. mapping means (8, 9, 13) for mapping image 
data output from said camera (2) onto a model produced 
in a three-dimensional coordinate system* 

c. transformation means (10, 11) for 
rotationally transforming said model with said image 
data mapped thereon in said three-dimensional coordinate 
system; and 

d. a monitor (14) for displaying said model 
with said image data mapped. thereon , characterized by 

e. producing, means (5, 6). based on 
coordinates of picture elements on the image pickup 
device of said camera (2) and the image pickup angle of 
said camera (2) with respect to said object (3), for 
producing said model representative of a surface. of said 
object (3) whose image has been picked up in said three- 
dimensional coordinate system. 
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2. The apparatus for processing image 
according to claim 1, characterized by 

a. measuring means (4) for measuring the 
image pickup angle^ of said camera with respect to said 
object. ' • 

% 

3. The apparatus for processing image 
according to claim 1, characterized by 

a. deformation means (11, 12) for deforming a 
specific portion of said model with said image data 
mapped thereon* 

4. The apparatus for processing image 
according to claim 1, characterized in that 

said producing means (5, 6) has planar model 
producing means (5) for producing a planar model by 
approximating to a plane the surface of said object (3) 
whose image has been picked up based on coordinates of 
picture elements on the image pickup device of said 
camera (2) and said image pickup angle and means (6) for 
modifying said planar model based on data representative 
of the height of said surface whose image has been 
picked up. 
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5 . . Apparatus constructed and arranged to operate 
substantially as hereinbefore described with reference to 
and as illustrated in the accompanying drawings. 
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